Add --db-index, --db-file, --taxonomy, --db-pipe for piped db files #89
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Allow kraken to operate on multiple inputs -> outputs
|
👍 It would be great to see this merged in. |
|
I'll take a look at this ASAP (hopefully today, if not by the end of this week) |
|
+1 Interested in this feature. Thanks! |
|
+1 also interested in this feature! |
|
@DerrickWood also interest on this end. |
|
@jenniferlu717 Any chance of a review or merge? |
|
Apologies....we've been working on releasing Kraken 2.0 ASAP and haven't had a chance to review this. It has a number of conflicts with the current code so merging is a bit more difficult but I will try to look at it soon..... |
|
In that case perhaps it's better for me to wait until 2.0 before resolving conflicts |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows loading the database from a pipe instead of a file. This is useful in cloud platforms like AWS/GCP to load the database from an object store download and decompress on the fly, which is both faster and cheaper than storing on networked disk.
Also allows processing multiple inputs in one run. This is more important for the loading from pipe scenario because the database files are not implicitly cached in the page cache between runs.